Search for: All records

Background and Objectives: The variability and biases in the real-world performance benchmarking of deep learning models for medical imaging compromise their trustworthiness for real-world deployment. The common approach of holding out a single fixed test set fails to quantify the variance in the estimation of test performance metrics. This study introduces NACHOS (Nested and Automated Cross-validation and Hyperparameter Optimization using Supercomputing) to reduce and quantify the variance of test performance metrics of deep learning models. Methods: NACHOS integrates Nested Cross-Validation (NCV) and Automated Hyperparameter Optimization (AHPO) within a parallelized high-performance computing (HPC) framework. NACHOS was demonstrated on a chest X-ray repository and an Optical Coherence Tomography (OCT) dataset under multiple data partitioning schemes. Beyond performance estimation, DACHOS (Deployment with Automated Cross-validation and Hyperparameter Optimization using Supercomputing) is introduced to leverage AHPO and cross-validation to build the final model on the full dataset, improving expected deployment performance. Results: The findings underscore the importance of NCV in quantifying and reducing estimation variance, AHPO in optimizing hyperparameters consistently across test folds, and HPC in ensuring computational feasibility. Conclusions: By integrating these methodologies, NACHOS and DACHOS provide a scalable, reproducible, and trustworthy framework for DL model evaluation and deployment in medical imaging. To maximize public availability, the full open-source codebase is provided at https://github.com/thepanlab/NACHOS. 

ABSTRACT A three‐dimensional convolutional neural network (3D‐CNN) was developed for the analysis of volumetric optical coherence tomography (OCT) images to enhance endoscopic guidance during percutaneous nephrostomy. The model was performance‐benchmarked using a 10‐fold nested cross‐validation procedure and achieved an average test accuracy of 90.57% across a dataset of 10 porcine kidneys. This performance significantly exceeded that of 2D‐CNN models that attained average test accuracies ranging from 85.63% to 88.22% using 1, 10, or 100 radial sections extracted from the 3D OCT volumes. The 3D‐CNN (~12 million parameters) was benchmarked against three state‐of‐the‐art volumetric architectures: the 3D Vision Transformer (3D‐ViT, ~45 million parameters), 3D‐DenseNet121 (~12 million parameters), and the Multi‐plane and Multi‐slice Transformer (M3T, ~29 million parameters). While these models achieved comparable inferencing accuracy, the 3D‐CNN exhibited lower inference latency (33 ms) than 3D‐ViT (86 ms), 3D‐DenseNet121 (58 ms), and M3T (93 ms), representing a critical advantage for real‐time surgical guidance applications. These results demonstrate the 3D‐CNN's capability as a powerful and practical tool for computer‐aided diagnosis in OCT‐guided surgical interventions. 

The variability and biases in the real-world performance benchmarking of deep learning models for medical imaging compromise their trustworthiness for real-world deployment. The common approach of holding out a single fixed test set fails to quantify the variance in the estimation of test performance metrics. This study introduces NACHOS (Nested and Automated Cross-validation and Hyperparameter Optimization using Supercomputing) to reduce and quantify the variance of test performance metrics of deep learning models. NACHOS integrates Nested Cross-Validation (NCV) and Automated Hyperparameter Optimization (AHPO) within a parallelized high-performance computing (HPC) framework. NACHOS was demonstrated on a chest X-ray repository and an Optical Coherence Tomography (OCT) dataset under multiple data partitioning schemes. Beyond performance estimation, DACHOS (Deployment with Automated Cross-validation and Hyperparameter Optimization using Supercomputing) is introduced to leverage AHPO and cross-validation to build the final model on the full dataset, improving expected deployment performance. The findings underscore the importance of NCV in quantifying and reducing estimation variance, AHPO in optimizing hyperparameters consistently across test folds, and HPC in ensuring computational feasibility. By integrating these methodologies, NACHOS and DACHOS provide a scalable, reproducible, and trustworthy framework for DL model evaluation and deployment in medical imaging. 

BackgroundTobacco use remains the leading cause of preventable mortality in the United States; yet, evidence-based cessation services remain underused due to staffing constraints, limited access to counseling, and competing clinical priorities. Generative artificial intelligence (GenAI) chatbots may address these barriers by delivering personalized, guideline-aligned counseling through naturalistic dialogue. However, little is known about how GenAI chatbots support smoking cessation at both outcome and communication process levels. ObjectiveThis feasibility study evaluated the implementation of an evidence-based smoking cessation counseling session delivered by a GenAI-powered chatbot, Aipaca. We examined (1) pre-post changes in cessation preparedness, (2) communication dynamics during counseling sessions, and (3) user perceptions of the chatbot’s value, limitations, and design needs. MethodsWe conducted an observational, single-arm, mixed methods study with 29 adult smokers. Participants completed pre-post surveys measuring knowledge of smoking-related health risks and cessation methods, self-efficacy, and readiness to quit. Each engaged in a 30-minute text-based counseling session with Aipaca, powered by GPT-4 and structured using the 5A’s framework (Ask, Advise, Assess, Assist, Arrange). Sessions were transcribed for microsequential conversation analysis. Twenty-five participants completed semistructured interviews exploring perceived value, challenges, and design suggestions. Quantitative data were analyzed with paired-samples t tests, qualitative data were thematically analyzed, and transcripts were analyzed for interactional practices. The methodological strength of this study lies in its triangulated approach, which combines quantitative measurement of intervention effectiveness, qualitative analysis of user interviews, and conversational analysis of counseling transcripts to generate a comprehensive understanding of both outcomes and underlying mechanisms. ResultsParticipants demonstrated significant improvements in all preparedness indicators: knowledge of health risks, knowledge of cessation methods, self-efficacy, and readiness to quit. Conversation analysis identified three recurrent patterns enabling counseling-relevant dynamics: (1) contextual referencing and continuity, (2) formulations with elaboration prompts, and (3) narrative progression toward collaborative planning. Interview themes underscored Aipaca’s perceived value as an accessible, nonjudgmental, and motivating resource, capable of delivering personalized and interactive support. Criticisms included limited accountability, reduced cultural resonance, and overly goal-directed style. Participants emphasized design needs such as proactive engagement, gamified progress tracking, empathetic or anthropomorphic personas, and safeguards for accuracy. ConclusionsThis mixed methods feasibility study demonstrates that GenAI can deliver evidence-based smoking cessation counseling with measurable short-term gains in cessation preparedness and process-level communication patterns consistent with motivational interviewing. Users valued Aipaca’s accessibility, empathy, and personalization, while also articulating expectations for richer social roles and long-term accountability. Findings highlight both the promise and challenges of integrating GenAI into digital health: pairing adaptive language generation with human-centered design, embedding accuracy safeguards, and ensuring integration into multilevel cessation infrastructures will be essential for future clinical deployment. 

Abstract Epidural anesthesia helps manage pain during different surgeries. Nonetheless, the precise placement of the epidural needle remains a challenge. In this study, we developed a probe based on polarization‐sensitive optical coherence tomography (PS‐OCT) to enhance the epidural anesthesia needle placement. The probe was tested on six porcine spinal samples. The multimodal imaging guidance used the OCT intensity mode and three distinct PS‐OCT modes: (1) phase retardation, (2) optic axis, and (3) degree of polarization uniformity (DOPU). Each mode enabled the classification of different epidural tissues through distinct imaging characteristics. To further streamline the tissue recognition procedure, convolutional neural network (CNN) were used to autonomously identify the tissue types within the probe's field of view. ResNet50 models were developed for all four imaging modes. DOPU imaging was found to provide the highest cross‐testing accuracy of 91.53%. These results showed the improved precision by PS‐OCT in guiding epidural anesthesia needle placement. 

Optical coherence tomography (OCT) imaging enables high resolution visualization of sub-surface tissue microstructures. However, OCT image analysis using deep learning is hampered by limited diverse training data to meet performance requirements and high inference latency for real-time applications. To address these challenges, we developed Octascope, a lightweight domain-specific convolutional neural network (CNN) - based model designed for OCT image analysis. Octascope was pre-trained using a curriculum learning approach, which involves sequential training, first on natural images (ImageNet), then on OCT images from retinal, abdominal, and renal tissues, to progressively acquire transferable knowledge. This multi-domain pre-training enables Octascope to generalize across varied tissue types. In two downstream tasks, Octascope demonstrated notable improvements in predictive accuracy compared to alternative approaches. In the epidural tissue detection task, our method surpassed single-task learning with fine-tuning by 9.13% and OCT-specific transfer learning by 5.95% in accuracy. Octascope outperformed VGG16 and ResNet50 by 5.36% and 6.66% in a retinal diagnosis task, respectively. In comparison to a Transformer-based OCT foundation model - RETFound, Octascope delivered 2 to 4.4 times faster inference speed with slightly better predictive accuracies in both downstream tasks. Octascope represented a significant advancement for OCT image analysis by providing an effective balance between computational efficiency and diagnostic accuracy for real-time clinical applications.